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Appendix A. Implementation Methods 


A.1 Interviews, Observations, and Document Analysis 


For this project, researchers conducted a total of 47 semi-structured interviews and focus groups 
with stakeholders at the three partner colleges, including administrators, math faculty who did not 
participate in the project, and all 22 math instructors who participated in the model development 
and pilot phases of the project. Faculty who served on the leadership teams, meaning they were 
integral to adapting the lesson study model for use in community colleges, were interviewed at the 
beginning of the project and again at the end. Interviews were transcribed and coded using Dedoose. 


The research team observed and took field notes during 10 lesson study cycles. Six of these cycles 
took place during the model development phase, while the other four took place in the pilot study 
phase. For pilot study cycles, researchers took field notes and scored each team’s cycle using an 
implementation fidelity rubric to assess the level of implementation across 18 indicators (see 
Appendix B). The rubric was developed with input from EdNW facilitators and from faculty who 
participated in the model development phase. Two researchers scored each of the four cycles 
independently, and the team saw high agreement across observers. In isolated incidents where 
scores differed, the average across observers was recorded. 


Across all 10 observed cycles, the research team collected the lesson plans for the teach and reteach 
sessions as well as each faculty team’s final report. For most but not all cycles, teams were working 
from original curricular materials, which were collected as examples of “baseline” instructional 
practices. These documents were analyzed qualitatively for the types of new instructional practices 
that teams adopted during the teach and reteach sessions. 


A.2 Faculty Survey Development, Administration, and Analysis 


To supplement the stakeholder interviews and further inform our second research question—Does 
lesson study influence instructors’ beliefs, curricular materials, and teaching practices?—the 
research team developed two faculty surveys. The pre-pilot, or baseline, version of the survey was 
administered in fall 2018, and the follow-up survey was administered in fall 2019. All mathematics 
faculty at the three colleges were invited by CCRC via email to participate (see Table A.1 for survey 
response rates). Both surveys collected information about instructors’ backgrounds (e.g., full-time or 
adjunct status, number of years teaching, courses taught, educational credentials), experiences 
working collaboratively with colleagues, and knowledge, beliefs, and self-efficacy in teaching 
mathematics, as well as about the professional development offerings at their colleges. The follow- 
up survey included an additional set of questions specifically for lesson study participants. Selected 
survey items were drawn and adapted from the Teaching and Learning International Survey 
(Organisation for Economic Co-operation and Development, 2018), the Teacher Self-Efficacy Scale 
(Tschannen-Moran & Hoy, 2001), and the TIMSS Teacher Questionnaire (Kastberg et al., 2013). Upon 
submission of a survey, respondents were entered into a lottery for a $50 gift card. 
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Faculty Survey Response Rates 


Fall 2018 Baseline Fall 2019 Follow-Up 


Total faculty population 241 236 
Survey respondents Til 104 
Survey response rate 46% 44% 


In order to assess the influence of participating in lesson study on instructors’ beliefs, teaching 
practices, and professional community, we restricted the faculty survey analytic sample to 
mathematics faculty who reported teaching at the developmental level within the past year. This 
approach yielded three samples for comparison: all 2018 developmental mathematics instructors (n 
= 90), 2019 lesson study participants (n = 22), and 2019 developmental mathematics instructors who 
did not participate in lesson study (n = 50). 


Neither version of the survey collected unique identifiers for respondents, therefore we were unable 
to assess changes in outcomes on an individual instructor basis from the pre-pilot to post-pilot 
periods. Instead, we tested for differences in survey item responses between the three groups. As 
the majority of survey items used a Likert scale of either 1 to 4 or 1 to 5, we first identified the 
proportion of respondents in each of the three groups who provided an affirmative response to the 
item (e.g., “agree”/“strongly agree” or “moderate confidence”/“high confidence”). We then 
conducted pairwise t-tests to determine whether the proportions of affirmative responses were 
statistically different. 


Appendix B. Lesson Study Implementation Rubric 
and Mean Pilot Scores 
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Cycle Stage 1: Study and Plan 


Plan a collaborative lesson 


Develop or revisit 
collaboration norms 


Develop or revisit the 
research theme 


Identify and study the topic 


Identify learning outcomes 
for the lesson 


Anticipate student 
responses and instructor 
support 


Apply evidence-based 
instructional approaches 


Establish points of 
evaluation 
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0) 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


2) 


Team meets to 
collaboratively plan the 
research lesson 

Team develops or revisits 
norms 


Team develops or revisits 
the research theme 


Team identifies where the 
lesson falls on the 
syllabus 


Team establishes 
learning outcomes for the 
lesson 


Team generates one or 
more anticipated student 
responses 


Team refers to evidence- 
based instructional 
approaches 


Team establishes one or 
more points of evaluation 


Cycle Stage 2: Teach, Observe, and Debrief 


Observe first teaching of the 


research lesson 


Debrief first teaching of the 


research lesson 


0) 


Not 
present 


Not 
present 


(72) 


Team members observe 
during the lesson period 


Team meets to debrief 
the lesson 


(3) 


Team completes (or almost 
completes) the lesson plan 
collaboratively 


Team refers to or explicitly uses 
norms during the planning process 


Team explicitly refers to the 
research theme during the 
planning process 

Team identifies a learning 


progression for the topic and the 
lesson’s place in that progression 


Team explicitly uses learning 
outcomes to guide instructional 
decisions during the planning 


process 


Team uses anticipated student 
responses to develop instructional 


support strategies 


Team provides an evidence- 
based rationale for instructional 
decisions during the planning 


process 
Team connects points of 


evaluation to learning outcomes, 
anticipated student responses, 


and/or research theme 


.C)) 


Team members take detailed 
observational notes focused on 


students and their learning 


Team follows the protocol, and all 
members share observational data 


during the debriefing 


Mean 
siexe) a=) 
25 


2.75 


2.75 


2.25 


2.5 


2.33 


2.33 


3 
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Cycle Stage 3: Revise and Reteach 


Revise the lesson 


Ground revision decisions in 


goals for students 


Apply evidence-based 
instructional approaches 


Observe second teaching of 


the research lesson 


Debrief second teaching of 


the research lesson 
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(1) 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Not 
present 


Cycle Stage 4: Reflect and Report 


Reflect and report 


Generate knowledge 


Share knowledge 


0) 


Not 
present 


Not 


present 


Not 
present 


(2) 


Team meets 
collaboratively to revise 
the research lesson 
Team refers to student 
learning outcomes and/or 
research theme during 
revision 


Team refers to evidence- 
based instructional 
approaches 


Team members observe 
during the lesson period 


Team meets to debrief 
the lesson 


7) 


Team meets to reflect on 
the cycle 


Team identifies 
implications for 
instruction beyond the 
research lesson 

Team discusses sharing 
knowledge learned during 
the cycle with a broader 
audience 


'€<)) 


Team completes (or almost 
completes) the lesson revisions 
collaboratively 

Team explicitly uses learning 
outcomes and/or the research 
theme to guide instructional 
decisions during the revision 
process 

Team provides an evidence-based 
rationale for instructional decisions 
during the revision process 


Team members take detailed 
observational notes focused on 
students and their learning 


Team follows the protocol, and all 
members share observational data 
during the debriefing 


'(<)) 


Team generates a report 
documenting their learning during 
the cycle 

Team articulates something new 
that they learned about 
instructional practice or student 
learning 

Team identifies a specific venue or 
format for sharing knowledge with 
faculty not on the lesson study 
team 


2.75 


2.5 


2.75 


2.25 


Appendix C. Student Learning Assessment 


C.1 Assessment Overview and Scoring 


To explore the effects of lesson study on student learning, we developed an in-class assessment on 
percentages, a key concept in the quantitative literacy pathway (see Figure C.1). 


Appendix Figure C.1 


Student Learning Assessment Items 


The table below shows the percentage of students by age at Shasta Community College in California 
for two different academic years. {Source: https://datamart.cccco.edu/students/Student Term Annual 


Count.aspx } 


2008 2009 


Student Count | Student Count 
(%) (%) 


100% 100% 


1. What observations can you make about changes in the student body from 2008 to 
2009? Use numbers to justify your claims. 


2. The total number of students at Shasta was 17,119 in 2008 and 15,406 in 2009. Which 
year (2008 or 2009) had more students aged 25 to 29? Explain your reasoning. 


Let's imagine that in the next year, in 2010, a recession resulted in a lot of layoffs at a 
local factory. The layoffs led to an increase in the number of students aged 40-49 going 
back to college. There were 1,540 students in 2009 and 3,578 students in 2010 in the 
40-49 age group. The other age groups did not change much. 


3. What would this increase in enrollment do to the percentage of 40—-49-year-old 
students? Explain your reasoning. 


4. How would this increase in enrollment change the percentages in the other categories? 
Explain your reasoning. 


Researchers collaborated with faculty leaders to develop criteria for identifying correct responses on 
each of the four items. Those criteria are listed in the table below. Two members of the research 
team test scored a sample of assessments and then shared those assessments and scores with the 
faculty leaders for review. After test scoring, the research team scored the remaining assessments. A 
selection of assessments (about 10%) were cross-scored by two members of the team to ensure 
inter-rater reliability. 
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Student Learning Assessment Scoring Rubric 


Student responses to Question 1 were e the student correctly identified the enrollment trends of 
identified as correct if: one or more specific age groups, 
OR 


e the student correctly identified one or more general 
enrollment trends. 


Student responses to Question 2 were e the student showed the correct number of students ages 
identified as correct if: 25 to 29 at Shasta Community College for both 2008 and 
2009 (both 2,054 and 2,055 were accepted for 2008, and 
both 2,002 and 2,008 were accepted for 2009), 
OR 
e the student correctly selected 2008 as having more 
students ages 25 to 29 than 2009 (even though the 
computations were not shown). 


Student responses to Question 3 were e the student correctly stated that the percentage of 
identified as correct if: students ages 40 to 49 would increase as a result of the 
increase in enrollment of this age group. 


Student responses to Question 4 e the student correctly stated that the percentages of other 
were identified as correct if: age categories would decrease as a result of the increase 
in enrollment of students ages 40 to 49. 


C.2 Technical Notes on Analytic Method for Student Learning Assessment 

Here we provide more technical details on the analysis of the student learning assessment 
administered to Math 098 students in fall 2018 (before the pilot phase) and fall 2019 (in the final 
weeks of the pilot phase). At one partner college, the assessment was also administered in sections 
of Math 058, a developmental course that serves as a prerequisite to Math 098 at some community 
colleges in Oregon. In total, over both years, 378 students completed the four-item assessment. 
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Student Learning Assessment Sample 


Fall 2018 Fall 2019 
Pre-Pilot Sections Lesson Study Non-Lesson Study 
rSYeYo Ho) als) Sections 
(Post-Pilot) (Post-Pilot) 
Math 058 42 81 30 
Math 098 139 39 47 
N 181 120 TT 


Students who completed the learning assessment during the fall 2019 term were asked to provide 
informed consent, which allowed the research team to match students’ learning assessment results 
with their administrative demographic records. The final, matched 2019 pilot sample included 197 
students, 61% of whom were enrolled in a section taught by a lesson study faculty participant. As 
Math 058 is a lower-level course, we analyzed the learning assessment outcomes for Math 058 and 
Math 098 students separately. 


We used two approaches to determine whether enrolling in a section taught by a lesson study 
participant influenced student performance on the learning assessment. First, we compared the 
performance of students in sections taught by participants (2019) with students in sections taught by 
nonparticipants (2019) and those in pre-pilot course sections (2018). To examine whether student 
performance differed across these three groups, we conducted a Pearson chi-square test for each of 
the five binary outcomes (i.e., a correct response on each of the four assessment items and 
answering all items correctly). Second, to control for underlying differences between students in 
sections taught by participants and those in sections taught by nonparticipants, we conducted 
binomial logistic regressions using the 2019 matched pilot sample that included students’ 
demographic characteristics, including race/ethnicity, gender, age, Pell Grant status, and prior 
enrollment. 


Math 098 learning assessment outcomes. In Table C.3, we show the aggregate percentage of 
students in Math 098 sections taught by lesson study participants, those in sections taught by 
nonparticipants, and those in pre-pilot sections who answered each learning assessment item 
correctly. We performed a chi-square test of independence for each outcome and found no 
significant differences in learning assessment performance between the three Math 098 groups. 


Appendix Table C.3 


Math 098 Learning Assessment Outcomes: Chi-Square Tests 


Lesson Study Non-Lesson 
Sections Study Sections Pre-Pilot 
eXeyre)) eXeyie)) Sections (2018) 

% correct % correct % correct Chi? p 
Item 1 79.5 80.9 79.9 0.03 985 
Item 2 46.2 51.1 56.8 1.57 455 
Item 3 79.5 74.5 81.3 1.01 .605 
Item 4 48.7 48.9 53.2 OA 814 
All correct 17.9 23.4 12.2 3.54 170 
N 39 47 139 


KES 01, *O< 05, *p< J 


Math 058 learning assessment outcomes. Table C.4 shows the proportion of correct responses and 
chi-square test results for students who completed the learning assessment while enrolled in Math 
058. 
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Math 058 Learning Assessment Outcomes: Chi-Square Tests 


Lesson Study Non-Lesson Pre-Pilot 
Sections Study Sections eneone 
(eZe}(2)) (Ze) (2) oXeyt=)) 

% correct % correct % correct Chi? p 
Item 1 66.7 73.3 85.7 512 O77 . 
Item 2 53.1 30.0 35.7 6.27 .043 ie 
Item 3 76.5 43.3 71.4 11.43 .003 wae 
Item 4 49.4 53.3 38.1 2.00 367 
All correct 22.2 20.0 48 6.22 .045 a 
N 81 30 42 


saa 0] < 01, *D < 05, *p < J 


Based on these chi-square tests, we were able to reject the null hypothesis for learning assessment 
items 2 and 3 as well as answering all items correctly. To estimate the influence of enrolling in a 
Math 058 section taught by a lesson study participant on learning assessment outcomes during the 
pilot term, we then omitted the 2018 comparison group and conducted multilevel regressions using 
only the two 2019 groups (students in sections taught by lesson study participants and 
nonparticipants). Table C.5 shows the results of mixed-effects logistic regression models for the five 
binary outcomes. All specifications in Table C.5 include student-level demographic fixed effects and 
random effects at the second, or instructor, level. We found a positive, statistically significant effect 
of enrolling in a Math 058 section taught by a lesson study participant for two assessment items. For 
items 2 and 3, we observe 1.09 (p = .035) and 1.38 (p = .010) changes in the log-odds of correctly 
answering these items, which correspond to estimated odds ratios of 2.97 and 3.97, respectively. 


Appendix Table C.5. 
Math 058 Learning Assessment Outcomes: Regressions 
0) 74) i) iC) (ey) 


(@) K@io)qg=\e] ma @ VA Ore) tn -\0] en @ 0 O10) an -\e] ME @- O10) ac-\ola rN M@re)aaz\eia 


Course taught by lesson study -0.785 1.089** 1.378*** -0.799 -0.463 
faculty (fall 2019) (0.572) (0.517) (0.532) (0.560) (0.648) 
Returning student -0.210 0.106 0.651 -0.226 -0.566 
(0.539) (0.508) (0.574) (0.554) (0.649) 
Black -0.568 0.122 -1.282 1199 113 
(0.970) (0.993) (1.041) (1.254) (1.329) 
Hispanic -0.531 -0.842 1.058 -0.0918 0.0457 
(0.684) (0.682) (0.905) (0.734) (0.842) 
Asian -1.724* -0.148 -2.373** -2.031* 
(0.949) (0.883) (1.184) (1.078) 
Multiracial -0.0218 -0.645 0.324 1.558 1.027 
(1.010) (0.996) (1.214) (1.093) (1.406) 
Female -0.635 -0.878* -0.802 -1.770*** -1.480** 
(0.480) (0.458) (0.527) (0.507) (0.652) 
Age (in years) 0.0515* 0.00900 0.000378 0.0523** 0.0747** 
(0.0290) (0.0236) (0.0258) (0.0253) (0.0309) 
Pell Grant recipient -0.623 0.0214 -0.432 -1.484*** -2.025*** 
(0.523) (0.474) (0.554) (0.541) (0.661) 
Constant 1.036 -0.523 0.0782 1.239 -0.983 
(0.905) (0.777) (0.830) (0.839) (0.932) 
Observations 108 108 108 aif@) 101 
Number of groups 5 5 5 5 5 
Instructor indicators Yes Yes Yes Yes Yes 


EOS 01, OS 05, *o< J 
Note: Standard errors in parentheses. 


Appendix D. Technical Notes on Analytic Method for Course 
Grades and Progression 


In the proposal for this project, the original research design for assessing student course outcomes 
was an interrupted time series (ITS). The ITS design would have been used to detect the impact of 
lesson study on students by comparing course outcomes in Math 098 after the intervention was 
implemented to the outcomes that might have been expected based on pre-intervention trends 
(using course data from fall 2015 to summer 2019). The difference between the actual and projected 
outcomes would provide an estimate of the causal impact of the intervention (Bloom, 2003). 
However, this design proved unsuitable for this project since only about half of the Math 098 faculty 
members participated in lesson study in fall 2019, and the design would have pooled outcomes 
across sections taught by participants and nonparticipants in fall 2019. ITS is typically used to assess 
the effectiveness of whole-school reforms (Bloom, 2003). Additionally, ITS requires a significant 
amount of post-treatment data to model post-treatment trends, and the project team only had two 
terms of post-treatment data (from fall 2019 and winter 2020). The third term (spring 2021) was 
affected by the COVID-19 pandemic when colleges transitioned to virtual instruction before final 
exams. For these reasons, we adapted the research design and decided to use propensity score 
weighting to assess the association between lesson study and Math 098 course grades and 
progression during the fall 2019 pilot term. 


Our sample included students taught by 15 faculty members in 18 sections of Math 098 in fall 2019. 
Eight of those faculty members participated in the lesson study pilot phase. Compared to students in 
sections taught by nonparticipants, students in sections taught by lesson study participants were 
slightly older on average and were more likely to be low-income (as measured by Pell Grant 
eligibility), female, and Hispanic (see Table D.1). Compared to students in sections taught by 
nonparticipants, students in sections taught by participants were also more likely to be returning 
students. Returning students who enrolled in sections taught by participants had previously 
attempted fewer developmental math credits and slightly more developmental reading/writing 
credits, on average, than returning students who enrolled in sections taught by nonparticipants. 
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Appendix Table D.1 


Characteristics of Students in Math O98 (Fall 2019) 


Number of Students 


Student Demographic Characteristics 


Student Enrollment Data 


Age 

Pell Grant recipient 

Veteran 

Female 

Male 

Gender not reported 

American Indian/Alaskan Native 
Asian 

Black 

Hispanic 

Multiracial 

Native Hawaiian/Pacific Islander 


White 
Race unknown 


Returning student 


Developmental reading/writing credits attempted 


Prior Developmental Coursework (Returning Students)* 


GPA in developmental education courses 


Developmental math credits attempted 


Developmental reading/writing credits attempted 
@ These variables are not included in the regression analysis described below. 
Note. Demographic differences between students in lesson study sections and non-lesson study sections were not 
statistically significant, except for age and Pell Grant recipient status. These differences were marginally significant at the 


10% level. 


Lesson Study 
ekexeuteyats) 


171 


21.6 
54% 
4% 
57% 
42% 
1% 
1% 
1% 
4% 
23% 
5% 
1% 


62% 
2% 


29% 
0.27 


2.88 
4.82 
0.94 


Non-Lesson Study 
Sections 


160 


20.5 
44% 
4% 
54% 
43% 
3% 
2% 
3% 
4% 
17% 
8% 
O% 


64% 
3% 


23% 
0.27 


2.84 
6.65 
0.84 


We matched students in sections taught by lesson study participants with students in sections taught 
by nonparticipants using radius matching that calculated a propensity score (the probability of being 
in a section taught by a participant in fall 2019) for each student in the sample based on the 
individual characteristics listed below in Table D.2. After calculating the propensity scores with a 
caliper of 0.1 and assigning weights to each student, we examined the propensity score distribution 
and standardized mean differences to determine whether there was sufficient overlap of the 
distribution of the propensity scores between the two groups, as well as successful balancing of the 


covariates across groups. For most covariates, the standardized difference in means was reduced 
between the unmatched and matched sample. This was especially important for the indicator of 
socioeconomic status: In the unmatched sample, 55% of students in sections taught by participants 
were Pell Grant recipients, compared to 45% of students in sections taught by nonparticipants. In the 
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matched sample, 54% of students in sections taught by participants were Pell Grant recipients, 
compared to 53% percent of students in sections taught by nonparticipants. 


Appendix Table D.2 
Characteristics of Matched and Unmatched Math 098 Students (Fall 2019) 


Students in Students in Standardized 
Student-Level Variables S¥-Taale)(c) Lesson Study Non-Lesson Difference in 
Sections Study Sections Group Means 
Unmatched 3% 4% 0.04 
plaek Matched 3% 3% 0.00 
Unmatched 23% 18% 0.13 
Hispanic Matched 22% 21% 0.01 
Unmatched 2% 3% 0.09 
Asian/Native Hawaiian Kictehed 2% 2% 0.02 
Unmatched 1% 2% O12 
American Indian/Alaskan Native Matched 0% 1% On 
Unmatched 6% 1% 0.06 
Mulireciel Matched 6% 1% 0.02 
Unmatched 3% 2% 0.04 
Race uuiniene Matched 3% 2% 0.03 
Unmatched 55% 54% 0.02 
Peitals Matched 54% 55% 0.03 
Unmatched 1% 2% 0.05 
Gender not reported Matohed 1% 1% 0.04 
Unmatched 4% 4% 0.01 
veteran Matched 4% 3% 0.03 
Unmatched 55% 45% 0.21 
Pell Grant recipient Matohed 54% 53% 0.03 
Unmatched 21.69 20.54 0.20 
Age Matched 21.09 20.55 010 
Unmatched 31% 24% 0.16 
Botunane.guident Matched 29% 25% 010 


Note. For balance to be achieved, the absolute standardized differences in means should be less than 0.25 (Stuart, 2010). 
The pre-match standardized differences in group means were between 0.01 and 0.21; after the matching procedure, the 
standardized differences in group means were between 0.00 and 0.10. The formula for calculating the standardized 
difference in group means is: 


aia, | 
((of + 0¢)/2) 


The study also used a multilevel model to account for college and instructor characteristics that 
could explain differences in outcomes between students in sections taught by participants and those 
in sections taught by nonparticipants. Since 12 of the 15 instructors taught only one section of Math 
098, it is also likely that students who had the same instructor were in the same section, meaning 
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that they shared characteristics about their course section that might be related to their outcomes, 
such as the peers in their classroom and the time of day that the course was held (Theobald, 2018). 
The model was: 


(1) Y;, =a+ BLesson Study;, + BXis + Vs + is 


In this model: 


Y = the outcomes of interest including grade in Math 098, passing the Math 098 course 
with C or better, persistence to the end of the Math 098 course (that is, they did not withdraw or 
have an incomplete grade), and enrollment in college-level math in the subsequent term (winter 
2020) 


Lesson Study = a dichotomous indicator that is equal to “1” if the student was enrolled in 
a Math 098 section taught by a faculty member participating in lesson study and “O” if the student 
was in a Math 098 section taught by a faculty member who was not participating in lesson study 


X = a vector of student characteristics (all variables listed in Table D.2) 
y =an indicator of community college (college fixed effects) 
€is = a random error term for student jin college s 


The full regression results for each outcome (course grade, passing the course, course persistence, 
and progression to college-level math) are shown in Table D.3. Each column (1) in Table D.3 shows 
the coefficient estimates from a linear regression with no matching, college fixed effects, or 
instructor random effects. Each column (2) in Table D.3 shows the final coefficient estimates from a 
regression analysis that used college fixed effects and instructor random effects to account for 
instructor impacts on student outcomes, as well as propensity score weights so that students in 
sections taught by nonparticipants that were similar to the students in sections taught by 
participants were weighted more heavily in the regression analysis. Propensity score analysis ensures 
that the treatment and comparison groups are comparable on the observed covariates, and 
covariate adjustment (through regression analysis) may then be able to reduce any remaining 
differences on the observed covariates between the two samples and improve the precision of the 
impact estimation (Rubin & Thomas, 2000). 


For course grade, we found a statistically significant negative impact of enrolling in a section taught 
by a lesson study participant in the model without matching (p = .02); the negative coefficient was no 
longer significant in the full model. For passing the Math 098 course, we also observed a negative 
effect (p < .01), which remained significant when propensity score weights and college and instructor 
effects were added (p = .01). For the other two course outcomes—persistence to the end of the 
course and progression to college-level math—no significant impact of enrolling in a section taught 
by a participant was found for either model specification. 
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Appendix Table D.3 


Math 098 Course Outcomes: Regressions 
a) 73) 0) 7) 0) 7) 0) (2) 


Grade Grade Pass Pass Persist Persist Progress Progress 
Course taught by lesson -0.35**  — -0.28 013" = -O1" = -0.01 0.00 0.01 -0.02 
study faulty (fall 2019) (015) (024) (0.05) (0.05) (0.02) (002) (0.05) (0.08) 
ae -0.45 -0.74 -018 014 -006  -0.04 -0.12 -0.09 
(0.41) (0.46) (0.14) (018) (0.08) (0.07) (0.12) (0.14) 
ee -0.02 -0.05 0.04 0.05 0.00 0.01 -0.01 0.02 
cine (019) (0.26) (0.06) (0.06) (0.02) (0.02) (0.07) (0.07) 
Rea ACen -0.48 -0.49* -O.N -012  -008  -010 -0.21* -0.07 
(044) ~— (0.30) (0.15) (0.1) (011) — (0.10) (0.12) (0.20) 

hich ai -0.24 0.59** -0.20 -0.15 0.02 0.04. -0.33*** -0.30*** 
aia (0.70) (0.28) (0.22) (022) (0.01) (0.03) (0.04) (0.06) 
eee -0.05 -0.05 0.05 0.01 -0.02  -0.05 0.05 0.09 
(0.31) (0.32) (0.10) (010) (0.05) (0.06) (0.11) (0.12) 
pe OsI** 0.49% 0.09* 0.08*  -000  -0.00 -0.06 -0.06 
(0.16) (0.12) (0.05) (0.05) (0.02) (0.01) (0.05) (0.04) 
Veer -0.29 -0.50 -0.09 -0.07  0.05* O.10* 0.07 -0.01 
(0.53) (0.33) (0.15) (012) (0.03) (0.04) (0.15) (0.10) 
Bach canicn 023 oso 0.07 0.07 0.00 -0.01 0.02 0.04 
P (0.16) (0.11) (0.05) (0.05) (0.02) (0.02) (0.06) (0.04) 

0.03*  0.03** -0.00 0.00  -000 -000* -0.00 -0.00 


fue ai tisetenm(Ve2's) (0.01) (0.01) (0.00) (0.00) (0.00) (000) (001) (000) 


-0.10 -0.24 0.06 0.02 0.03 0.02 -0.05 0.01 


Returning suuertt (018) (017) (0.06) (0.06) (002) (002) (0.06) (0.07) 


Dev. reading/writing credits -0.17* -0.26*** -0.03 -0.04 0.01** 0.03** -0.00 -0.04* 
(fall 2019) (0.09) (0.07) (0.03) (0.04) (0.00) (0.01) (0.03) (0.02) 
1.85*** 2.47*** 0.75*** 0.73*** = =1.02*** 0.94*** 0.43*** 0.59*** 
Constant (0.32) (0.47) (0.09) (0.12) (0.05) (0.07) (0.11) (0.10) 
Observations 310 303 331 326 331 326 331 326 
R-squared 0.10 0.06 0.06 0.03 
College indicators No Yes No Yes No Yes No Yes 
Instructor indicators No Yes No Yes No Yes No Yes 
Matched group No Yes No Yes No Yes No Yes 
Number of groups 


*** 0 < O1,**0<.05, *p< A 


Note: Asian and Native Hawaiian/Pacific Islander combined to account for small sample size in the Native Hawaiian/Pacific 
Islander group. Robust standard errors in parentheses. 
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